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LONG-TERM  GOALS 

To  predict  the  probability  distribution  function  (pdf)  of  medium  range  weather  forecast  errors  as 
accurately  as  possible. 

OBJECTIVES 

Objective  1:  To  compare  the  Bishop  et  al.’s  (2001)  recently  developed  Ensemble  Transform  Kalman 
Filter  (ET  KF)  ensemble  generation  technique  against  the  breeding  of  growing  vectors  (BGV) 
technique  (Toth  and  Kalnay,  1993,  1997)  in  a  GCM. 

Objective  2:  To  quantify  the  limits  of  an  ET  KF  ensemble  that  does  not  explicitly  account  for  model 
error  to  predict  forecast  error  variance  in  a  GCM. 

Objective  3:  To  identify  and  remove  (a)  model  error  bias,  (b)  model  error  that  correlates  with 
variations  in  key  parameters  controlling  the  model’s  parameterizations  of  unresolved  processes  and  (c) 
model  error  that  correlates  with  deviations  of  the  model  trajectory  about  the  climatological  mean. 
(NWP  failure  to  predict  cold  air  damming  due  to  poorly  resolved  topography  is  a  fine  example  of  a 
systematic  model  error  that  would  correlate  with  the  deviation  of  the  model  trajectory  about  the 
climate  mean.) 

Objective  4:  To  create  and  test  an  ensemble  generation  scheme  that  accounts  not  only  for  the  loss  of 
predictability  due  to  initial  condition  error  but  also  for  the  loss  of  predictability  due  to  model  error. 

APPROACH 

The  above  aims  are  motivated  by  the  fact  that  in  current  operational  ensemble  prediction  systems,  e.g., 
the  singular  vector  method  (Buizza  and  Palmer  1995;  Molteni  et  al.  1996)  adopted  by  the  European 
Centre  for  Medium-Range  Weather  Forecasting  (ECMWF),  the  breeding  method  (Toth  and  Kalnay 
1993,1997)  used  at  National  Centers  for  Environmental  Prediction  (NCEP),  the  ratio  of  ensemble 
variance  to  forecast  error  variance  diminishes  with  time  form  the  3  day  to  10  day  forecast  lead  time. 
Work  by  Houtekamer  et  al.  (1996)  and  Smith  (2001)  makes  it  clear  that  a  major  reason  for  this 
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deficiency  in  ensemble  forecasts  is  the  presence  of  model  error.  As  such,  model  error  is  a  major  issue 
in  quantifying  predictability. 

To  build  up  stable  statistics  for  model  error,  one  needs  to  examine  many  years  worth  of  data.  The 
limitations  of  the  computing  resources  we  expect  to  obtain  for  this  study  thus  forces  us  to  restrict  our 
study  to  the  predictability  characteristics  of  a  model  with  considerably  lower  resolution  than  the 
models  typically  used  for  NWP. 

WORK  COMPLETED 

Considerable  progress  has  already  been  made  in  achieving  Aims  1  and  2.  Xuguang  has  developed 
numerical  tools  to  run  T42  CCM3  ensemble  forecasts  off  the  NCEP/NCAR  reanalysis  data  set  on  4 
PCs  each  with  dual  933  MHz  processes  running  Linux.  She  has  already  begun  using  these  tools  to 
compare  the  performance  of  ensemble  perturbations  during  the  2000  Boreal  summer  (JJA)  generated 
using  the  breeding  method  (Toth  and  Kalnay,  1997)  with  ensemble  perturbations  generated  using  the 
recently  developed  ET  KF  ensemble  generation  technique. 

In  this  new  ET  KF  ensemble  generation  scheme,  forecast  perturbations  listed  as  columns  in  the  matrix 
Zf  are  transfonned  into  analysis  perturbations  Za  by  a  transformation  matrix  T,  that  is,  Za=  Z1  T.  These 
analysis  perturbations  are  then  added  to  the  analysis  to  give  the  initial  conditions  for  the  subsequent 
ensemble  forecast.  The  transfonnation  matrix  T  is  chosen  in  order  to  ensure  that  the  covariance  matrix 
associated  with  the  transformed  perturbations  Fa  =  Za  ZaT  would  be  precisely  equal  to  the  true 
analysis  error  covariance  Pat  if  F1  =  Z1  Z1T  were  precisely  equal  to  the  true  forecast  error  covariance 
matrix  Ptf . 

In  the  breeding  technique,  all  of  the  forecast  ensemble  perturbations  are  transformed  into  analysis 
perturbations  by  multiplying  each  of  them  by  a  constant  factor  whose  magnitude  is  less  than  one.  Thus, 
in  its  simplest  fonn,  the  breeding  technique  takes  no  account  of  variations  in  observational  density  nor 
does  it  account  for  the  fact  that  data  assimilation  schemes  reduce  error  in  directions  corresponding  to 
large  forecast  error  variance  more  than  directions  corresponding  to  small  forecast  error  variance  (cf 
Daley,  1991).  Because  the  breeding  method’s  transformation  from  forecast  perturbations  to  analysis 
perturbations  reduces  perturbation  amplitude  in  all  directions  by  the  same  factor,  directions 
corresponding  to  slowly  growing  errors  maybe  removed  from  the  ensemble  perturbation  subspace. 

Indeed,  if  the  atmosphere  went  into  a  quasi-stationary  state,  all  bred  perturbations  would  eventually 
take  on  the  characteristics  of  the  fastest  growing  eigenvector  of  the  perturbation  dynamics  propagator 
associated  with  the  quasi-stationary  basic  state.  In  this  case,  all  of  the  perturbations  would  be 
approximately  parallel  to  each  other  and  there  would  be  little  point  in  having  more  than  one  or  two 
ensemble  members. 

In  contrast,  the  ET  KF  transformation  of  forecast  perturbations  into  analysis  perturbations  accounts  for 
variations  in  observational  density.  Furthermore,  consistent  with  filtering  properties  of  an  optimal  data 
assimilation  scheme,  it  ensures  that  perturbation  amplitude  is  reduced  more  in  directions 
corresponding  to  large  forecast  error  variance  than  it  is  in  directions  corresponding  to  small  forecast 
error  variance.  These  considerations  led  to  the  following  hypotheses. 
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Hypothesis  1 .  The  variance  of  initial  ET  KF  ensemble  members  would  better  reflect  inhomogeneities 
in  analysis  error  due  to  inhomogeneities  in  the  observational  network  than  the  corresponding  variance 
of  initial  bred  mode  ensemble  members. 

Hypothesis  2.  The  spectrum  of  eigenvalues  of  the  forecast  error  covariance  matrices  produced  by  the 
ET  KF  ensemble  will  be  much  flatter  than  the  corresponding  spectrum  of  eigenvalues  produced  by  the 
bred-mode  ensemble;  i.e.  the  ET  KF  ensemble  will  produce  ensemble  spread  in  many  more  directions 
than  the  bred-mode  ensemble. 

Hypothesis  3.  Bred-vector  forecast  errors  are  more  highly  correlated  than  ET  KF  ensemble  members. 

RESULTS 

To  test  these  hypotheses,  Xuguang  ran  an  8  member  ET  KF  T42  CCM3  ensemble  for  the  2000  Boreal 
summer  and  compared  the  characteristics  of  this  ensemble  with  the  characteristics  of  an  8  member 
bred  mode  ensemble  over  the  same  period.  For  the  ET  KF  ensemble  generation  scheme,  it  was 
assumed  that  the  observational  network  consisted  solely  of  rawinsondes  released  every  12  hours.  For 
both  the  breeding  and  ET  KF  techniques,  Dee’s  (1995)  maximal  likelihood  parameter  estimation 
theory  was  used  to  ensure  that  12  hr  ensemble  perturbation  magnitude  was  consistent  with  12  hr 
forecast  error  at  rawinsonde  sites.  Fig.  1  compares  the  seasonal  mean  vertically  averaged  ensemble 
wind  variance  of  ensemble  members  at  the  analysis  time  for  the  breeding  technique  (Fig.  la)  and  the 
ET  KF  technique  (Fig.  lb).  First,  note  that  initial  perturbation  amplitude  in  the  observation  scarce 
southern  hemisphere  is  much  larger  for  the  ET  KF  than  it  is  for  the  breeding  technique.  Second,  note 
that  despite  the  high  concentration  of  rawinsondes  over  the  Eurasian  continent,  initial  bred 
perturbation  amplitude  is  locally  maximized  in  this  region.  In  contrast,  ET  KF  initial  perturbation 
amplitude  is  quite  small  in  this  region.  These  characteristics  of  Fig.  1  are  consistent  with  hypothesis  1. 

Gross  characteristics  of  Fig.  1  that  are  not  clearly  consistent  with  hypothesis  1  are  that  localized 
concentrations  of  rawinsonde  observations  such  as  those  in  South  Africa  and  South  America  had  no 
perceptible  effect  on  mean  ET  KF  initial  ensemble  perturbation  amplitude.  Moreover,  we  are 
concerned  that  while  there  is  a  local  mid-latitude  minimum  in  perturbation  amplitude  over  rawinsonde 
dense  North  America,  initial  perturbation  amplitude  seems  unrealistically  high.  Since  with  an 
ensemble  of  only  8  members  there  is  a  limited  number  of  observation  density  characteristics  towards 
which  the  ET  KF  perturbations  can  adjust,  presumably  these  aspects  of  the  ET  KF  ensemble  analysis 
variance  would  be  reduced  if  a  larger  ensemble  were  used.  Tests  are  currently  under  way  to  test  this 
presumption. 

Fig.  2  compares  the  seasonal  mean  spectrums  of  eigenvalues  of  the  ensemble  based  12  hr  forecast  error 
covariance  matrices  for  the  bred-mode  ensemble  and  the  ET  KF  ensemble. 
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(a)  Bred  Ensemble 


(b)  ET  KF  Ensemble 


Fig.  1  Seasonal  mean  vertically 
averaged  ensemble  wind 
variance 
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Figure  2.  Mean  ETKF  and 
BGV  eigenvalues 
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In  confinnation  of  hypothesis  2,  Fig.  2  shows  that  the  spectrum  of  ET  KF  eigenvalues  is  much  flatter 
than  the  bred-mode  eigenvalues.  In  other  words,  while  there  are  large  amounts  of  ensemble  forecast 
variance  present  in  all  seven  orthogonal  directions  of  the  ET  KF  ensemble  nearly  all  of  the  bred-mode 
ensemble  forecast  variance  is  contained  in  a  single  direction. 

In  confirmation  of  Hypothesis  3,  the  average  error  correlation  of  T42  forecasts  of  2m  temperatures  at 
northern  hemisphere  mid-latitude  rawinsonde  sites  for  ET  KF  and  breeding  members  was  found  to  be 
0.79  and  0.91,  respectively.  Presumably,  the  relatively  large  error  correlations  found  for  both 
techniques  is  due  to  model  error. 

IMPACT/APPLICATIONS 

At  NRL  Monterey,  research  is  being  conducted  to  improve  FNMOC’s  (bred  vector)  ensemble 
forecasting  capabilities.  Because  of  the  positive  results  found  in  our  preliminary  tests,  the  ETKF 
ensemble  generation  scheme  and  other  schemes  will  be  tested  at  NRL  to  determine  their  suitability  for 
transition  into  operations  at  FMNOC.  Zoltan  Toth  and  Mohzeng  Wang  of  the  National  Centers  for 
Environmental  Prediction  (NCEP)  are  also  preparing  to  test  versions  of  the  ETKF  ensemble  generation 
scheme  to  determine  its  suitability  as  a  replacement  to  their  current  bred  vector  scheme. 

TRANSITIONS 

NCEP,  in  collaboration  with  former  Post-doctoral  fellow  Sharanya  Majumdar,  graduate  student  Brian 
J.  Etherton  and  undergraduate  student  Jonathon  Moskaitis,  is  currently  applying  the  ETKF  to  a 
combined  ECMWF/NCEP  ensemble  to  determine  were  aircraft  should  fly  in  the  ongoing  NOAA 
Winter  Storms  Reconnaissance  program. 

RELATED  PROJECTS 

The  NSF  grant  ATM-98- 14376  “Adaptive  Sampling  with  the  Ensemble  Transform  Kalman  filter” 
enabled  tests  of  the  ability  of  the  ETKF  to  predict  reductions  in  forecast  error  variance  due  to  targeted 
observations.  See  http://www.met.psu.edu/dept/faculty/bishop.htm  and 
http://orca.rsmas.miami.edu/~majumdar/  for  details. 

SUMMARY 

In  order  to  more  accurately  represent  the  uncertainty  in  weather  forecasts  a  new,  computationally 
inexpensive  method  has  been  devised  for  generating  multiple  forecasts  whose  differences  reflect 
weather  forecast  uncertainty.  Our  tests  indicate  that  the  method  is  superior  to  the  breeding  technique 
that  is  currently  used  by  the  federally  funded  civilian  and  Naval  weather  forecasting  agencies. 
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